Discriminative Learning Can Succeed Where Generative Learning Fails
نویسندگان
چکیده
Generative algorithms for learning classifiers use training data to separately estimate a probability model for each class. New items are classified by comparing their probabilities under these models. In contrast, discriminative learning algorithms try to find classifiers that perform well on all the training data. We show that there is a learning problem that can be solved by a discriminative learning algorithm, but not by any generative learning algorithm. This statement is formalized using a framework inspired by previous work of Goldberg [4].
منابع مشابه
Efficient Heuristics for Discriminative Structure Learning of Bayesian Network Classifiers
We introduce a simple order-based greedy heuristic for learning discriminative structure within generative Bayesian network classifiers. We propose two methods for establishing an order of N features. They are based on the conditional mutual information and classification rate (i.e., risk), respectively. Given an ordering, we can find a discriminative structure with O ( Nk+1 ) score evaluations...
متن کاملDiscriminative Training of a Neural Network Statistical Parser
Discriminative methods have shown significant improvements over traditional generative methods in many machine learning applications, but there has been difficulty in extending them to natural language parsing. One problem is that much of the work on discriminative methods conflates changes to the learning method with changes to the parameterization of the problem. We show how a parser can be t...
متن کاملHybrid Deep Discriminative/Generative Models for Semi-Supervised Learning
Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpolating between the two extremes with a multiconditional likelihood object...
متن کاملDiscriminative Learning of Generative Models for Sequence Classification and Motion Tracking
I consider the issue of learning generative probabilistic models (e.g., Bayesian Networks) for the problems of classification and regression. As the generative models now serve as target-predicting functions, the learning problem can be treated differently from the traditional density estimation. Unlike the likelihood maximizing generative learning that fits a model to overall data, the discrim...
متن کاملLearning Discriminative Metrics via Generative Models and Kernel Learning
Metrics specifying distances between data points can be learned in a discriminative manner or fromgenerative models. In this paper, we show how to unify generative and discriminative learning of met-rics via a kernel learning framework. Specifically, we learn local metrics optimized from parametricgenerative models. These are then used as base kernels to construct a global kerne...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Inf. Process. Lett.
دوره 103 شماره
صفحات -
تاریخ انتشار 2006